An efficient virtual network interface in the FUGU scalable workstation dc by Kenneth Martin Mackenzie

A scalable workstation is one vision of a mainstream parallel computer: a machine that combines scalable, fine-grain communication facilities for parallel applications with virtual memory and preemptive multiprogramming to support general-purpose workloads. A key challenge in a scalable workstation is the Virtual Network Interface (VNI) problem. The problem is that high performance communication for parallel programming depends on a tight coupling between the application and the network while multiprogramming and virtual memory effects disrupt such coupling. This thesis introduces and evaluates the “direct” virtual network interface: a solution to the VNI problem for fine-grain messages in a scalable workstation. The direct VNI employs two complementary architectural techniques to reconcile speed and protection. First, two-case delivery optimistically provides direct, user-level access to network interface hardware but also transparently backs the direct system with a robust, software-buffered system. Two-case delivery allows the scalable workstation to support both good parallel application performance through the fast hardware interface and good global system performance by permitting buffering when required for multiprogramming. Second, the software-buffered mode uses virtual buffering to provide effectively unlimited buffer capacity by storing messages in dynamically managed virtual memory. Virtual buffering gives the user the convenient illusion of a very large buffer while giving the operating system the means to minimize actual, physical memory consumption. The direct VNI ideas are implemented in an experimental scalable workstation, FUGU, consisting of emulated hardware, a matching simulator and a custom operating system. Results from workloads of real and synthetic applications show that the direct VNI provides high performance because the direct case is both fast and common. Microbenchmarks show the protected direct delivery case costs only 60% (10s of cycles per message) more than unprotected messages on the same hardware. Further, in a mixed workload experiment, we observe that our parallel applications see only 14 33% of messages buffered when 10% of the CPU time is devoted to unrelated, high-priority, interactive tasks. Finally, results show that physical buffering requirements remain naturally low in real applications despite the combination of unacknowledged messages and unlimited buffering. Thesis Supervisor: Anant Agarwal Title: Associate Professor of Computer Science and Engineering Thesis Supervisor: M. Frans Kaashoek Title: Associate Professor of Computer Science and Engineering